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United  States 

General  Accounting  Office 

Washington,  D.C.  20548 


Human  Resources  Division 
B-224648 

December  3,  1987 

The  Honorable  Andrew  Jacobs,  Jr. 
Chairman,  Subcommittee  on  Social  Security 
Committee  on  Ways  and  Means 
House  of  Representatives 

Dear  Mr.  Chairman: 


In  response  to  a  request  from  the  former  Chairman  and  subsequent  dis¬ 
cussions  with  the  Subcommittee,  we  have  been  monitoring  how  well  the 
Social  Security  Administration’s  (ssa’s)  demonstration  projects  are  being 
implemented  and  evaluated.  On  several  occasions  we  briefed  your  office 
on  their  status.  This  briefing  report  presents  the  results  of  our  work  in 
more  detail. 


f 

I 

I 


>  The  demonstration  projects,  mandated  by  the  Social  Security  Disability 
Benefits  Reform  Act  of  1984  (Public  Law  98-460),  involve  face-to-face 
interviews  for  claimants,  including  applicants  at  the  initial  stage  in  the 
disability  determination  (decision)  process  and  current  beneficiaries 
during  continuing  disability  reviews.  These  interviews,  at  state  agencies 
for  Disability  Determination  Services,  take  place  before  a  final  decision 
is  made.  The  projects  are  intended  to  test  whether  these  early  face-to- 
face  interviews  result  in  more  accurate  evaluations  of  an  applicant’s 
condition,  assure  that  all  relevant  information  is  obtained,  and  simplify 
and  expedite  the  decision-making  process.  Currently,  face-to-face  inter¬ 
views  between  applicants  and  examiners  do  not  take  place  (1)  until  a 
decision  is  made  final  and  (2)  generally,  unless  there  is  an  appeal  to  an  y 

administrative  law  judge.  . .  lj  j  % 

~ .  "■  — u 

We  began  our  work  in  March  1986  and  completed  field  work  in  May 
1987.  We  discussed  implementation  of  the  demonstration  projects  with 
officials  from  ssa  and  the  ten  participating  states;  we  also  reviewed  the  □ 

analysis  plan,  developed  by  a  contractor  that  ssa  selected  to  evaluate  □ 

the  demonstration  projects.  In  addition,  we  reviewed  all  California  disa- - - 

bility  cases  where  claimants,  during  the  second  quarter  of  1986,  were  * 

offered  a  face-to-face  interview  at  the  reconsideration  level  of  the  - J 

appeals  process.  Although  the  interviews  were  not  part  of  the  demon- 
stration  projects,  we  believe  the  results  provide  some  insight  on  how  ^.1°? 
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Demonstration 
Projects  Start  Slowly 


GAO  Concerns  About 
SSA’s  Demonstration 


ssa  was  required  by  law  to  report  to  the  Congress  by  December  1986  on 
the  projects’  results.  Two  problems  contributed  to  delays  in  implementa¬ 
tion:  (1)  Developing  and  obtaining  top  management  approval  for  pro¬ 
gram  regulations  and  procedures,  including  forms  to  record  information, 
took  longer  than  anticipated.  (2)  ssa  experienced  delays  in  contracting 
for  the  projects’  evaluation.  The  initial  contract  was  awarded  in  August 
1985,  but  subsequently  terminated  because  the  Department  of  Health 
and  Human  Services  Inspector  General  determined  that  ssa  had  not 
requested  the  approval  of  the  Secretary  of  Health  and  Human  Services 
before  the  award.  On  July  3, 1986,  a  new  evaluation  contract  was 
awarded  to  the  same  contractor.  In  April  1987,  this  contract  was  termi¬ 
nated,  according  to  ssa  officials,  because  the  contractor  failed  to  do  an 
adequate  job.  The  Inspector  General  is  now  responsible  for  evaluating 
the  projects. 

In  a  March  3,  1987,  interim  report  to  the  Congress,  the  Commissioner  of 
ssa  indicated  that  a  final  report  would  not  be  completed  until  September 
1988.  Due  to  recent  changes  in  the  evaluation  plans,  however,  this  com¬ 
pletion  date  could  be  delayed. 


To  provide  the  Congress  with  quantitative  data  on  the  costs  and  effec¬ 
tiveness  of  adopting  new  appeals  procedures,  ssa  should  assure  that  the 
demonstration  is  carefully  and  consistently  implemented  and  evaluated. 
ssa  needs  to  identify  the  financial,  legal,  and  procedural  impacts  of 
adopting  the  demonstration  process,  ssa  should  assess  the  extent  to 
which  states  reach  better  determinations  earlier  in  the  disability  deter¬ 
mination  process  as  a  result  of  demonstration  procedures. 

In  a  February  19,  1987,  report  to  the  Commissioner  of  ssa,'  we  pointed 
out  a  number  of  factors  that  could  prevent  the  demonstration  from 
meeting  its  objectives.  We  stated  that  it  may  not  be  possible  for  ssa  to 
adequately  assess  these  factors  for  several  reasons: 

States  are  not  implementing  the  projects  on  a  consistent  basis,  which 
may  bias  the  results. 

Demonstration  results  cannot  be  statistically  representative  of  all  state 
Disability  Determination  Services.  In  addition,  states  participating  in  the 
demonstration’s  initial  application  portion  have  a  relatively  low  volume 


'Social  Snuritj  Demonstration  Project*  Concerning  Interviews  With  Disability  Claimants  it . At) 

iirorsr -  -  — ■  * 


Page  2 


GAO,  HKD-A&-22BR  SSA  Demonstration  Interviews 


B- 224648 


of  initial  applications;  thus,  the  data  could  provide  misleading  results  on 
the  effectiveness  of  the  projects. 

•  The  demonstration  results  will  be  compared  against  those  for  eviden¬ 
tiary  hearings;  because  both  are  operationally  new  processes,  the  learn¬ 
ing  curves  of  each  could  affect  the  comparability  of  outcomes. 

•  It  might  not  be  possible  to  separate  a  claimant’s  satisfaction  with  the 
process  and  satisfaction  with  the  outcome;  there  is  no  statistical  basis  to 
believe  that  the  data  from  the  states  for  a  claimant  satisfaction  survey 
will  represent  claimants’  knowledge,  understanding,  or  satisfaction 
nationwide. 

In  a  May  29,  1987,  letter  the  ssa  Commissioner  generally  agreed  with 
our  recommendations  for  addressing  these  concerns  (see  app.  I).  The 
Commissioner  said  the  demonstration  projects  will  be  evaluated  within 
states,  taking  into  consideration  how  one  state’s  operations  differ  from 
those  of  other  states.  She  also  said  ssa  would  work  closely  with  the 
Office  of  the  Inspector  General  to  evaluate  the  extent  to  which  the 
learning  curve  affects  key  outcome  measures  in  the  projects.  The  Com¬ 
missioner  was  uncertain,  however,  whether  an  evaluation  of  the  mea¬ 
sure  of  claimant  satisfaction  would  take  place. 

The  Commissioner  added  that  given  the  variations  between  states,  no 
single  random  sample  of  five  states  could  adequately  represent  the 
nation;  therefore,  states  were  selected  to  represent  the  range  of  charac¬ 
teristics  of  all  states  in  the  nation.  According  to  the  Commissioner,  the 
representativeness  of  nationwide  statistics  will  depend  on  how  varied 
the  results  are  between  the  five  demonstration  states.  From  discussions 
with  statistical  experts,  we  have  concluded  that  regardless  of  the  consis¬ 
tency  of  state  data,  ssa’s  data  will  not  be  statistically  representative  of 
the  nation. 


During  1985,  before  the  current  demonstration,  California  participated 
in  a  reconsideration  interview  project,  which  offered  face-to-face  inter¬ 
views  at  the  reconsideration  level  of  the  appeals  process  to  some  initial 
applicants.  Our  analysis  of  a  group  of  these  cases  showed  that  (1)  face- 
to-face  interviews  at  this  level  did  not  reduce  appeal  or  reversal  rates  at 
the  administrative  law  judge  level,  (2)  whether  or  not  the  applicant 
actually  appeared  at  the  interview  had  little  impact  on  the  appeal  or 
reversal  rates,  and  (3)  applicants  may  not  be  presenting  their  cases  in 
the  best  light  and  at  the  earliest  opportunity. 


California’s  Earlier 
Experience  Provides 
Insight 
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Conclusions 


Any  changes  to  the  current  system  should  be  carefully  considered  and 
should  have  merits  that  would  justify  revising  it.  These  merits — such  as 
more  accurate  decisions  at  a  lower  level — should  be  objectively  mea¬ 
sured  during  the  evaluation  of  the  projects.  Although  we  have  concern 
about  whether  the  data  can  be  statistically  representative  of  the  nation, 
we  believe  ssa’s  proposed  actions  are  generally  responsive  to  our 
recommendations. 


As  requested  by  your  office,  we  did  not  obtain  agency  comments  on  a 
draft  of  this  report.  However,  we  did  discuss  its  contents  with  agency 
program  officials  and  incorporated  their  views  where  appropriate. 
Unless  you  announce  its  contents  earlier,  we  plan  no  further  distribution 
of  this  report  until  30  days  from  its  issue  date.  At  that  time,  we  will 
send  copies  to  the  Secretary  of  Health  and  Human  Services  and  will 
make  copies  available  to  others  on  request. 

Should  you  need  any  additional  information  on  the  contents  of  this 
report,  please  call  me  on  275-6193. 


Sincerely  yours, 


Page  4 


GAOHRD-88-22BR  SSA  Demonstration  Interviews 


GAO/HRD4M-22BR  S8A  Demon* (ration  Interview* 


Social  Security  Disability:  Observations  on 
Demonstration  Interviews  With 


Disability  Claimants 

Q  - fr  * 


Introduction 


Disability  decisions  are  made  by  54  “state”  agencies,1  which  are  known 
as  Disability  Determination  Services  (DDSs).  The  disability  determination 
process,  which  is  essentially  the  same  for  both  Social  Security  Disability 
Insurance  (ssdi)  and  Supplemental  Security  Income  disability  and  blind¬ 
ness  claims,  can  involve  determinations  (decisions)  at  five  distinct 
levels — (1)  initial  claim  or  continuing  disability  review  (cdr)  decision, 
(2)  first  appeal:  reconsideration,  (3)  second  appeal:  administrative  law 
judge  (au),  (4)  third  appeal:  Appeals  Council,  and  (5)  fourth  appeal: 
civil  action.  At  least  two  of  these  decision  levels  provide  due  process 
protection  to  claimants,-  in  accordance  with  the  Constitution  and  a  1969 
Supreme  Court  decision. 1  ^ 


Examiners  and  medical  consultants  (physicians)  at  DDSs  review  disabil¬ 
ity  applications  and  make  initial  disability  decisions.  If  the  initial  deci¬ 
sion  is  a  denial,  the  applicant  can  request  to  have  that  decision  reviewed 
by  another  examiner.  This  process  is  known  as  “reconsideration.”  No 
face-to-face  interviews  take  place  between  applicants  and  examiners  at 
the  initial  level  or  the  reconsideration  level.  If  the  applicant  is  found 
ineligible  at  the  reconsideration  level,  the  decision  may  be  appealed  to 
an  au. 


In  addition  to  making  the  initial  decision,  the  DDS  examiners  periodi¬ 
cally  review  the  eligibility  of  those  on  the  disability  rolls.  These  cdrs  are 
required  by  law.  If  the  beneficiary  is  not  found  to  be  disabled  during  the 
cdr  process,  a  formal  notice  is  sent  indicating  why  and  advising  the  ben¬ 
eficiary  of  his  or  her  appeal  rights. 

In  1983,  the  Congress  changed  the  cdr  process  by  enacting  a  provision  in 
Public  Law  97-455  requiring  that  beneficiaries  being  dropped  from  the 
disability  rolls  be  given  an  opportunity  to  continue  receiving  their  bene¬ 
fits  until  they  have  had  an  evidentiary  hearing.4  Although,  typically,  in 
the  Social  Security  disability  programs,  evidentiary  hearings  have  only 


'One  in  each  state  ( except  South  Carolina,  which  has  a  separate  agency  for  the  blind),  the  District  of 
Columbia.  Guam,  and  Puerto  Rico 

-In  this  report,  a  claimant  is  referred  to  as  an  "applicant  '’  in  an  initial  claim  case  and  as  a  "benefici¬ 
ary"  if  already  on  the  disability  mils  If  both  applicants  and  beneficiaries  are  referred  to,  "claimant" 
is  the  term  used 

'in  Goldberg  v  Kelly  397  I'.S  254  ( 1970).  the  court  held  that  a  person's  statutory  entitlement  to 
welfare  benefits  was  a  form  of  property  interest  that  could  not  be  terminated  without  offering  a 
hearing  that  satisfied  specific  requirements  of  due  process 

4The  opportunity  afforded  bonoficiari<‘s  for  introducing  evidence  and  being  represented  by  counsel 
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been  held  by  aus,  such  hearings  are  also  now  being  held  at  the  reconsid¬ 
eration  level.  At  this  level  of  the  disability  decision,  each  beneficiary  is 
to  be  given  the  opportunity  to  present  his  or  her  case  to  a  DDS  hearing 
officer.  The  process  for  initial  applicants  and  beneficiaries  being 
reviewed  under  cdr  is  outlined  in  figure  1. 

In  October  1984,  the  Congress  enacted  the  Social  Security  Disability 
Benefits  Reform  Act  (Public  Law  98-460),  which,  among  other  things, 
required  ssa  to  conduct  demonstration  projects  providing  face-to-face 
interviews  between  examiners  and  applicants  or  beneficiaries  during  the 
decision-making  process  at  ddss.  These  interviews  are  to  be  conducted  at 
the  initial  decision  level  (for  both  initial  applicants  and  beneficiaries) 
before  a  final  unfavorable  decision  is  made  by  a  DDS.  The  demonstra¬ 
tion  projects  are  intended  to  test  the  effectiveness  of  face-to-face  inter¬ 
views:  Will  face-to-face  interviews  (at  the  initial  decision  level  and 
during  cdrs)  result  in  more  accurate  evaluations  of  a  claimant’s  condi¬ 
tion,  assure  that  all  relevant  information  is  obtained,  and  simplify  and 
expedite  the  decision-making  process?  For  cdr  beneficiaries,  the  demon¬ 
stration  differs  from  the  evidentiary  hearings  in  that  demonstration 
interviews  occur  before  a  final  decision  is  reached;  on  the  other  hand, 
the  evidentiary  hearings  are  part  of  an  appeal  and  occur  after  the  bene¬ 
ficiary  has  been  informed  that  he  or  she  is  no  longer  considered  eligible 
for  disability  benefits. 


Objectives,  Scope,  and 
Methodology 


The  former  Chairman,  House  Subcommittee  on  Social  Security,  Commit¬ 
tee  on  Ways  and  Means,  requested  that  we  evaluate  several  issues  per¬ 
taining  to  the  Social  Security  disability  programs.  Specifically,  we  were 
requested  to  observe  how  well  the  face-to-face  interview  demonstration 
projects  are  being  implemented  and  evaluated.  We  discussed  implemen¬ 
tation  with  officials  from  ssa  and  participating  states  (Arizona,  Califor¬ 
nia,  Florida,  Maine,  Michigan,  Mississippi,  Missouri,  New  Jersey,  New 
Mexico,  and  Washington),  and  reviewed  the  scope  of  work  and  the  anal¬ 
ysis  plan  developed  by  ssa’s  evaluation  contractor. 


We  began  our  work  in  March  1986  and  in  May  1987  completed  our  field 
work  at  ssa  headquarters  in  Baltimore,  ssa's  regional  office  in  San  Fran¬ 
cisco,  and  the  California  Disability  Evaluation  Division  in  Sacramento, 
California.  California  was  selected  because  it  was  the  only  state  to  actu¬ 
ally  start  processing  demonstration  cases  in  1986. 

To  obtain  information  on  the  implementation  of  the  state  demonstration 
projects,  we  sent  questionnaires  to  all  the  states  planning  to  participate. 
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Since  then,  six  of  these  states  (Alabama,  Georgia,  Kansas,  Montana, 
Nebraska,  and  Texas)  have  withdrawn  from  the  projects  and  three 
new  states  (Florida,  New  Mexico,  and  Washington)  have  been  added. 
Where  state  responses  were  unclear  or  raised  additional  questions, 
we  telephoned  program  directors.  To  obtain  more  current  data  on 
implementation  of  the  projects,  we  telephoned  each  state  during  May 
1987,  when  each  had  had  at  least  2  months  of  experience  with  the 
projects.  To  determine  whether  the  evaluation  of  the  projects  was 
likely  to  be  useful,  we  reviewed  ssa’s  contract  file  for  the  evaluation 
contract,  the  scope  of  work  for  the  evaluation,  and  a  draft  of  the 
contractor’s  analysis  plan. 

To  gain  insight  on  how  face-to-face  interviews  might  affect  the  Social 
Security  disability  programs,  we  examined  case  files  in  California, 
where  applicants  participated  in  a  special  interview  project  in  1985:  271 
initial  applicants  were  offered  face-to-face  interviews  at  the  reconsider¬ 
ation  level.5  These  cases  represent  all  cases  where  a  reconsideration 
interview  was  offered  and  a  decision  made  final  between  April  1  and 
June  30,  1985.  Although  the  selection  methodology  used  by  California 
was  not  random,  we  believe  these  cases  offer  some  insight  into  what 
might  be  expected  from  using  face-to-face  interviews  in  the  demonstra¬ 
tion  projects. 


Most  states  began  implementing  the  face-to-face  interviews  in  January 
1987,  more  than  2  years  after  the  legislation  for  demonstration  projects 
was  enacted.  Because  of  delays  in  implementation,  ssa  did  not  meet  the 
deadline  of  December  1986  mandated  for  the  final  report  to  the  Con¬ 
gress.  In  a  March  1987  interim  report  to  the  Congress,  ssa  officials  esti¬ 
mated  the  final  report  would  not  be  completed  until  September  1988. 

ssa  began  planning  for  the  demonstration  projects  in  October  1984, 
when  it  sent  a  memorandum  to  all  regional  commissioners  (1)  outlining 
how  the  demonstration  projects  would  operate  and  (2)  requesting  their 
comments.  Proposed  rules  were  circulated  internally  in  January  1985, 
and  participating  states  were  selected  in  March  1985.  Even  though  ssa 
began  planning  the  demonstration  projects  shortly  after  passage  of  the 
legislation,  implementation  has  not  moved  forward  smoothly;  between 


Demonstration 
Projects  Off  to  a  Slow 
Start 


r'These  interviews  were  held  at  the  suggestion  of  SSA  officials  so  that  DDS  hearing  officers  could 
maintain  their  skills  during  the  moratorium  on  CDRs.  California  was  1  of  15  states  that  offered  inter¬ 
views  during  1985. 
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Contracting  for  the 
Evaluation 


January  1985  and  January  1987,  ssa  continually  postponed 
implementation. 

Several  problems  contributed  to  the  delays  in  implementation:  ssa 
imposed  a  moratorium  on  CDRs  between  April  1984  and  December  1985. 
This  moratorium  was  a  result  of  court  decisions  and  significant  confu¬ 
sion  over  the  proper  criteria  for  terminating  benefits  of  those  currently 
on  the  disability  rolls.  In  1984,  amendments  to  Public  Law  98-460 
required  ssa  to  develop  a  medical  improvement  review  standard  for  ter¬ 
minating  benefits.  This  standard  was  developed  and  the  regulatory  pro¬ 
cess  completed  late  in  1985.  During  this  period,  ssa  concentrated  its 
efforts  on  developing  the  standard  necessary  to  begin  processing  cdr 
cases  and  accomplished  little  with  its  demonstration  projects.  In  Janu¬ 
ary  1986,  ssa  authorized  the  states  to  resume  the  cdr  process. 

ssa  also  experienced  delays  in  obtaining  approvals  for  the  implementa¬ 
tion  of  the  projects.  For  example,  ssa  initially  planned  to  issue  final  pro¬ 
gram  regulations  for  the  projects  by  May  1985.  The  regulations  were  not 
published,  however,  until  April  1986.  According  to  the  projects  director, 
this  delay  resulted  primarily  from  delays  in  obtaining  approvals  from 
the  Department  of  Health  and  Human  Services  (hhs)  and  the  Office  of 
Management  and  Budget.  In  addition,  before  states  could  implement  the 
projects,  ssa  had  to  develop  and  distribute  a  predecision  notification 
form  for  the  demonstration  test  group  (see  p.  14).  This  form  notifies 
claimants  that  a  preliminary  decision  has  been  made  to  deny  or  termi¬ 
nate  their  benefits  and  informs  them  of  their  rights  to  (1)  present  addi¬ 
tional  evidence  of  disability  and  (2)  meet  in  a  face-to-face  interview 
with  the  examiner  making  the  decision,  ssa  did  not  complete  develop¬ 
ment  of  this  predecision  form  until  September  1986,  when  forms  were 
issued. 


ssa  also  experienced  problems  contracting  for  the  evaluation  of  the  dem¬ 
onstration.  The  initial  contract  was  not  awarded  until  August  1985  and 
was  subsequently  terminated  because  it  had  not  been  awarded  in  com¬ 
pliance  with  hhs  requirements.  The  new  contract,  awarded  almost  a  year 
later,  was  terminated  in  April  1987  because  the  contractor  failed  to  do 
an  adequate  job. 

Although  ssa  has  planned  to  manage  the  demonstration  projects,  it  con¬ 
tracted  with  Advanced  Sciences  Incorporated  (asi)  to  conduct  the  over¬ 
all  evaluation.  According  to  ssa  officials,  an  outside  contractor  was 
selected  for  several  reasons,  including  ( 1 )  lack  of  in-house  expertise 
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required  to  measure  public  reaction,  (2)  maximizing  the  impartiality  and 
credibility  of  the  evaluation,  (3)  helping  to  assure  a  more  objective  reac¬ 
tion  from  those  being  interviewed,  and  (4)  providing  for  a  more  timely 
evaluation.  The  original  contract  was  awarded  to  asi  on  August  29, 

1985.  In  September  1985,  however,  the  contract  was  terminated  because 
the  hhs  Office  of  the  Inspector  General  (oig)  determined  that  it  was  a 
consultant  contract  and  should,  therefore,  have  been  approved  by  the 
Secretary  of  hhs  prior  to  award.  On  January  17,  1986,  ssa  requested  the 
Secretary  to  give  it  authority  to  contract  for  consultant  services; 
approval  was  granted  in  April  1986.  On  the  basis  of  this  approval,  ssa 
reinitiated  the  procurement  process.  A  new  contract,  substantially  the 
same  as  the  earlier  contract,  was  awarded  on  July  3,  1986,  to  the  same 
contractor  that  received  the  initial  award;  the  contract  was  for  $1.5  mil¬ 
lion,  about  $50,000  more  than  the  original  contract.  In  April  1987,  ssa 
once  again  terminated  the  evaluation  contract.  According  to  ssa,  the 
most  recent  termination  took  place  because  the  contractor  failed  to  meet 
the  specified  deadlines  and  claimed  significant  cost  overruns  while  pre¬ 
paring  the  analysis  plan  and  developing  the  survey  questionnaire. 

Currently,  ssa  intends  to  have  oig  evaluate  the  projects.  As  of  September 
29, 1987,  OIG  officials  had  not  developed  an  analysis  plan.  In  addition, 
oig  and  SSA  officials  had  not  reached  a  decision  on  whether  a  claimant 
satisfaction  survey  would  be  included  in  the  evaluation.  According  to 
oig  officials,  if  claimant  satisfaction  was  included,  it  would  be  a  mail 
survey  as  opposed  to  the  in-person  interview  planned  by  asi. 

From  the  originally  planned  implementation  date  of  May  1985,  the  dem¬ 
onstration  projects  were  delayed  more  than  20  months.  At  the  December 
1986  reporting  deadline  to  the  Congress,  ssa  had  little  actual  experience 
with  the  demonstration.  In  the  remainder  of  this  report,  we  discuss  how 
ssa  expects  the  demonstration  projects  to  work,  and  we  summarize  our 
concerns  with  the  demonstration.  In  addition,  we  summarize  our  review 
of  a  special  interview  project,  mentioned  earlier.  We  also  brought  our 
concerns  about  the  demonstration  to  the  attention  of  the  ssa  Commis¬ 
sioner  in  a  previous  report.’’  The  Commissioner  responded  to  the  report 
in  a  May  29,  1987,  letter,  and  her  comments  are  included  in  this  report, 
where  appropriate  (see  app.  I). 


’’ Demonstration  Projects  Concerning  Interviews  With  Disability  Claimants  (GAO/HRD-87-35,  Feb. 
1571587 ). 
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The  law  for  the  face-to-face  interview  demonstration  projects  requires 
the  participation  of  at  least  five  states  for  initial  claim  cases  and  five 
states  for  cdr  cases.  The  five  for  initial  claims  are  Arizona,  Michigan, 
Mississippi,  New  Mexico,  and  Washington;  the  five  for  cdrs  are  Califor¬ 
nia,  Florida,  Maine,  Missouri,  and  New  Jersey. 

ssa  guidelines  for  the  demonstration  projects  provide  for  randomly  plac¬ 
ing  cases  in  either  a  test  or  control  group.  In  the  test  group,  after  initial 
review  of  the  evidence  and  a  preliminary  decision  to  deny  or  terminate 
benefits,  claimants  will  be  offered  an  interview  with  the  examiner  decid¬ 
ing  his  or  her  case.  A  claimant  will  be  allowed  to  present  additional  evi¬ 
dence,  bring  along  representatives,  and  provide  testimony  at  the 
interview  about  his  or  her  disability. 

After  the  interviews,  examiners  will  make  the  final  determinations.  If 
benefits  are  denied  or  terminated,  the  next  level  of  appeal  will  be  to  an 
au.  For  test-group  claimants,  this  face-to-face  interview  will  replace  the 
reconsideration  level.  Control-group  claimants  will  go  through  the  usual 
ssa  appeals  procedures  for  initial  applicants  and  beneficiaries.  An  evalu¬ 
ation  will  be  made  of  the  disability  determination  process  for  claimants 
in  the  control  and  test  groups.  The  data  will  be  analyzed  to  determine 
whether  the  demonstration  procedures  result  in  more  accurate  and  less 
costly  decisions,  fewer  appeals,  and  a  more  expeditious  decision-making 
process  overall  than  the  control  procedures. 


In  order  for  ssa  to  meet  its  objective — providing  the  Congress  with 
quantitative  data  on  the  costs  and  effectiveness  of  adopting  new 
appeals  procedures  based  on  face-to-face  interviews  earlier  in  the  deci¬ 
sion-making  process — ssa  will  have  to  assure  that  the  demonstration  is 
carefully  and  consistently  implemented  and  evaluated.  The  demonstra¬ 
tion  and  evaluation  should  address  the  expected  results  of  changes  to 
the  current  system.  These  results  (mentioned  above)  should  be  ade¬ 
quately  defined  and  assessed  in  ssa’s  final  report. 

In  our  report  to  the  ssa  Commissioner,  we  stated  that  it  may  not  be  pos¬ 
sible  for  ssa  to  adequately  assess  such  results  for  these  reasons: 

•  Several  states  indicated  that  they  are  not  planning  to  implement  the 
projects  in  accordance  with  ssa  instructions.  These  differences  may  bias 
the  projects’  results.  (In  a  response  to  our  report,  the  ssa  Commissioner 
stated  that  under  the  current  federal-state  arrangement,  states  have  a 
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certain  amount  of  flexibility  in  administering  and  managing  their  opera¬ 
tions.  If  demonstration  procedures  are  implemented  nationwide,  how¬ 
ever,  DDSs  would  not  have  the  flexibility  to,  for  example,  exclude  certain 
populations  from  the  interview  process.  The  demonstration  should 
implement  procedures  that  closely  approximate  those  that  would  be 
adopted  if  the  process  was  adopted  nationwide.  According  to  the  ssa 
Commissioner,  the  state  projects  will  be  evaluated  by  taking  into  consid¬ 
eration  the  state  ddss’  operational  differences.) 

•  Demonstration  results  will  not  be  statistically  representative  of  all  ddss. 
In  addition,  because  states  participating  in  the  demonstration’s  initial 
application  portion  have  a  relatively  low  volume  of  initial  applications, 
the  data  could  provide  misleading  results  on  the  projects’  operational 
effectiveness.  (This  situation  has  not  changed  since  our  report  to  the  ssa 
Commissioner.  According  to  her,  the  representativeness  of  the  data 
nationwide  will  depend  on  how  varied  the  results  are  between  the  five 
project  states.  According  to  statistical  experts,  regardless  of  the  consis¬ 
tency  of  state  data,  ssa  will  not  be  able  to  draw  a  conclusion  that  is  sta¬ 
tistically  representative  of  the  nation.) 

•  cdr  demonstration  results  will  be  compared  against  the  evidentiary 
hearings  process,  both  operationally  new  processes.  (The  Commissioner 
responded  that  SSA  would  work  closely  with  oig  to  evaluate  the  extent  to 
which  the  learning  curve  has  an  impact  on  key  measures  of  outcome  in 
the  projects.) 

•  Evaluating  claimant  satisfaction  is  a  difficult  task  because  it  may  not  be 
possible  to  separate  claimants’  satisfaction  with  the  process  and  satis¬ 
faction  with  the  outcome.  There  is  no  statistical  basis  to  believe  that  a 
claimant  satisfaction  survey  for  the  states  would  represent  claimants’ 
knowledge,  understanding,  and  satisfaction  nationwide.  In  addition,  ful¬ 
filling  this  portion  of  the  evaluation  contract  is  costly.  As  noted  earlier, 
ssa  and  oig  are  currently  uncertain  whether  such  an  evaluation  will  take 
place  because  evaluating  claimant  satisfaction  would  be  difficult  and 
costly. 

In  May  1987  we  contacted  the  states  participating  in  the  projects  to 
update  our  information  on  implementation.  We  found  that  only  Califor¬ 
nia  (a  cdr  demonstration  state)  and  Michigan  (an  initial  application  dem¬ 
onstration  state)  said  they  would  complete  the  required  number  of  cases 
(5,000  cases  for  a  cdr  state  and  6,000  for  an  initial  application  state)  by 
the  September  30,  1987,  deadline  set  by  ssa.  On  July  17,  1987,  ssa  offi¬ 
cials  told  us  that  two  states  participating  in  the  initial  application  dem¬ 
onstration  will  be  given  a  30-day  extension  so  that  at  least  three  states 
will  complete  the  desired  initial  caseload.  In  addition,  ssa  is  giving  three 
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states  participating  in  the  cdr  demonstration  a  30-day  extension  so  that 
more  cases  can  be  completed. 

ssa  officials  said  that  for  the  cdr  demonstration,  ssa  will  not  have 
enough  cases  completed  to  achieve  the  necessary  400  cases  appealed  to 
the  alj  level,  ssa’s  current  data  show  that  only  about  100  to  125  cases 
will  get  to  the  alj  level,  which  will  require  ssa  to  further  qualify  the  cdr 
results. 

With  ssa’s  approval  in  most  cases,  states  are  using  varying  approaches 
to  implement  the  projects.  For  example,  eight  states  are  using  a  two- 
examiner  approach:  one  makes  the  initial  decision  and  another  conducts 
the  interview  and  prepares  the  final  decision.  In  two  states,  the  exam¬ 
iner  who  makes  the  initial  decision  also  holds  the  interview  and  makes 
the  final  decision.  In  two  other  states,  some  claimants  are  not  offered 
the  opportunity  to  participate  in  the  projects  because  they  do  not  live 
near  the  state  DDS  or,  in  the  case  of  Arizona,  because  they  are  Indians. 
According  to  a  state  official,  these  claimants  are  precluded  from  the 
projects  because  it  would  be  too  costly  to  include  them  (for  example,  the 
state  officials  said  that  interpreters  would  be  needed  for  the  Indians). 


During  1985,  before  the  current  demonstration,  California  participated 
in  a  reconsideration  interview  project,  mentioned  earlier.  The  process 
used  to  conduct  these  interviews  wa«  generally  comparable  with  that  of 
the  demonstration,  except  the  scope  of  the  project  was  smaller.  For 
example,  fewer  applicants  were  offered  interviews  and  only  ssdi  appli¬ 
cants  were  included  in  the  California  project.  According  to  state  offi¬ 
cials,  with  the  exceptions  noted  above,  all  cases  were  eligible  for 
interviews. 

All  333  case  files  in  which  face-to-face  interviews  were  offered  between 
April  and  June  1985  were  considered  for  review.  We  selected  this  period 
to  provide  sufficient  time  for  most  cases  appealed  to  the  alj  level  to  be 
completed.  Some  of  these  cases  were  excluded  from  our  analysis,  how¬ 
ever,  for  one  of  the  following  reasons: 

No  alj  decision  had  been  reached  at  the  conclusion  of  our  field  work  (14 

cases). 

The  case  was  remanded,  without  a  decision,  to  be  reevaluated  by  the 
state  under  new  criteria  for  mental  impairments  (29  cases). 
ssa  could  not  find  the  claimant’s  case  file  during  the  period  of  our 
review  (19  cases). 


Page  16 


GAO/HRD-88-22BR  SSA  Demonstration  Interviews 


Social  Security  Disability:  Observations  on 
Demonstration  Interviews  With 
Disability  Claimants 


Face-To-Face  Interviews 
Had  Insignificant  Impact 
on  Appeal  or  Reversal 
Rates 


Our  observations  are  based  on  the  remaining  271  cases.  Recognizing  the 
limitations  of  our  sample,  the  results,  nonetheless,  provide  some  insights 
on  how  face-to-face  interviews  affected  the  decision-making  process  in 
California  during  our  3-month  test  period. 


ssa  and  state  officials  generally  expected  or  hoped  that  face-to-face 
interview's  with  initial  applicants  during  the  decision-making  process 
would  produce  more  accurate  decisions  and  better  satisfy  applicants. 
Although  it  is  costly  and  more  time-consuming  to  have  such  interviews 
earlier  in  the  process,  it  was  expected  to  lower  the  rate  of  appealed  deci¬ 
sions  and,  further,  to  result  in  appealed  decisions  being  upheld  by  aus. 

This  did  not  happen  in  the  cases  we  reviewed.  About  72  percent  of  the 
applicants  denied  at  reconsideration  appealed  to  the  aljs.  This  is 
slightly  higher  than  the  67  percent  estimated  by  ssa’s  Office  of  Hearings 
and  Appeals  for  such  appeals  in  fiscal  year  1985.  ssa  officials  did  not 
know  why  there  was  such  a  high  rate  of  appeal.  In  addition,  whether  or 
not  the  applicant  actually  appeared  at  the  interview  had  little  impact  on 
either  the  au  appeal  or  reversal  rate.  For  example,  68  percent  of  our 
sample  applicants  who  did  not  attend  the  face-to-face  interview 
appealed  to  the  aus;  72  percent  of  those  applicants  who  did  attend  the 
interviews  appealed  to  the  aus. 

The  au  reversal  rate  was  73  percent  (1 1 1  applicants)  for  those  appli¬ 
cants  who  appealed  after  the  face-to-face  interview  at  the  reconsidera¬ 
tion  level.  (The  outcomes  for  our  sample  cases  are  summarized  in  fig.  2.) 
In  contrast,  during  the  first  quarter  of  fiscal  year  1986,  aus  reversed 
about  64  percent  of  all  California  disability  decisions  appealed  to  them. 
Nationwide,  in  fiscal  year  1985,  aus  reversed  about  55  percent  of  the 
cases  appealed.  We  could  not  determine  why  applicants  offered  an  inter¬ 
view'  had  a  higher  au  reversal  rate  than  applicants  who  were  not 
offered  such  an  interview. 
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Figure  2:  Outcomes  of  Face-to-Face 
Interviews  of  Sample  Cases  in  California 
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ssa  officials  said  that  a  possible  reason  for  the  higher  reversal  rate  on 
the  cases  we  reviewed  was  that  aus  concluded  the  applicant’s  residual 
functional  capacity  (RFC)  should  be  reduced,  thus  making  the  claimant 
eligible  for  benefits.  RFC  is  a  measure  of  the  applicant’s  ability  to  per¬ 
form  work  and  is  supposed  to  be  prepared  by  a  physician  who  reviews 
the  case  file,  ssa  officials  noted  that  during  the  reconsideration  inter¬ 
views,  if  the  hearing  officer,  based  on  the  face-to-face  interview,  wanted 
to  reduce  the  claimant’s  rfc,  agreement  had  to  be  reached  with  the  phy¬ 
sician.  If  the  physician  refused,  the  rfx;  could  not  be  changed.  Physicians 
were  not  present  at  the  interviews  and  based  their  decisions  solely  on 
evidence  in  applicants’  files.  This  is  the  same  review  procedure  that  will 
be  used  in  the  demonstration  projects. 

We  noted  some  differences  between  the  reconsideration  interviews  and 
the  alj  hearings,  which  could  contribute  to  the  outcomes  discussed 
above.  For  example,  only  4  percent  of  the  claimants  were  represented  by 
an  attorney  at  the  reconsideration  level,  but  85  percent  of  the  claimants 
had  such  representation  at  the  au  level.  Similarly,  although  45  claim¬ 
ants  (21  percent)  waived  or  failed  to  show  up  for  the  reconsideration 
interviews,  only  8  (5  percent)  missed  an  alj  hearing.  In  some  cases,  it 
appeared  that  the  reconsideration  interview  was  waived  on  advice  of 
counsel.  In  over  80  percent  of  the  appeals  reversed  at  the  au  level,  the 
claimant  submitted  additional  evidence  not  provided  at  the  reconsidera¬ 
tion  interview.  In  over  30  percent  of  the  cases  we  reviewed  in  detail,  this 
additional  evidence  appeared  to  be  a  significant  factor  in  the  au  deci¬ 
sion.  For  example,  in  one  case,  evidence  provided  to  the  hearing  officer 
was  general,  consisting  primarily  of  hospital  records.  The  additional  evi¬ 
dence  presented  to  the  au  more  specifically  addressed  the  disabling  con¬ 
dition,  including  reports  from  the  treating  physicians  and  laboratory 
tests.  Where  additional  evidence  had  an  impact  on  the  au  decision,  in 
only  3  of  the  10  cases  the  evidence  clearly  related  to  a  new  or  changed 
disabling  condition.  This  further  demonstrates  that  applicants,  for 
whatever  reason,  may  not  be  presenting  their  cases  in  the  best  light  and 
at  the  earliest  opportunity. 

Finally,  our  review  in  California  indicates  that  providing  a  face-to-face 
interview  for  initial  applicants  may  be  quite  costly.  During  our  3-month 
test  period,  interviewing,  scheduling,  and  clerical  work  for  all  reconsid¬ 
eration  interviews  averaged  almost  7  hours  a  case.  Reconsideration 
averaged  about  3  hours  a  case  without  the  offer  of  an  interview  and  the 
corresponding  scheduling  and  decision-writing  requirements.  During  our 
3-month  test  period,  15,774  initial  applicants  filed  for  reconsideration  in 
California.  If  over  80  percent  of  these  applicants  accepted  the  offer  of  a 
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face-to-face  interview,  as  was  the  case  for  our  sample,  we  estimate  it 
would  have  necessitated  an  increase  of  at  least  17  staff  years.  This  esti¬ 
mate  does  not  include  any  travel  costs,  loss  of  productivity  during 
travel,  or  office  space  costs  in  remote  areas. 


In  evaluating  the  demonstration  projects,  it  is  important  to  remember 
what  purpose  the  Social  Security  disability  appeal  process  serves — 
namely,  to  provide  due  process  protection  for  people  whose  disability 
benefits  have  been  denied  or  terminated.  The  current  appeal  procedures 
provide  this  protection.  In  fact,  for  beneficiaries  undergoing  cdrs,  at 
least  two  hearings  are  available. 

Any  changes  to  the  current  system  should  be  carefully  considered, 
objectively  demonstrating  merits  that  would  justify  revising  this  system. 
These  merits — such  as  more  accurate  and  timely  decisions  at  a  lower 
level — should  be  objectively  measured  during  the  projects’  evaluation 
process. 

We  had  several  concerns  about  the  implementation  of  the  projects, 
which  could  limit  the  results  of  the  evaluation,  and  we  reported  on  them 
to  the  ssa  Commissioner.  Although  we  still  have  concern  that  the  data 
are  not  statistically  representative  of  the  nation,  we  believe  ssa’s  stated 
actions  are  generally  responsive  to  our  recommendations.  In  addition, 
our  review  of  face-to-face  interviews  held  in  California  during  1985 
showed  that  the  interviews  did  not  lower  the  number  of  cases  appealed 
nor  the  number  of  decisions  reversed  by  the  aljs.  Although  the  inter¬ 
views  held  in  California  were  generally  comparable  with  those  in  the 
demonstration,  the  scope  of  the  earlier  project  was  more  limited. 


Palp  20 


(•AO  HRD-H8-22BR  SSA  Demonalration  Interview* 


Comments  From  the  Social  Security 
Administration  on  GAO/HRD-87-35 


THE  COMMISSIONER  OF  SOCIAL  SECURITY 
BALTIMORE  MARYLAND  21235 


m  29  1987- 


Mr.  Joseph  F.  Delfico 

Senior  Associate  Director 

Human  Resources  Division 

United  States  General  Accounting  Office 

441  G  Street,  N.W.,  Room  6739 

Washington,  D,C.  20548 

Dear  Mr.  Delfico: 

Enclosed  is  our  response  to  the  report.  X f  we  can  be  of 
further  assistance,  please  let  us  know. 


of  Social  Security 
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COMMENTS  OF  THE  SOCIAL  SECURITY  ADMINISTRATION  ON  THE  GENERAL 
ACCOUNTING  OFFICE'S  FINAL  REPORT.  "SOCIAL  SECURITY— DEMONSTRATION 
PROJECTS  CONCERNING  INTERVIEWS  WITH  DISABILITY  CLAIMANTS* 


General 

The  Social  Security  Administration  (SSA)  recently  terminated 
the  demonstration  projects  evaluation  contract.  Instead,  the 
Department  of  Health  and  Human  Services'  Office  of  Inspector 
General  (OIG)  will  perform  the  evaluation.  The  issue  of  whether 
and  how  to  include  measurement  of  client  satisfaction  in  the 
overall  evaluation  will  be  considered  as  the  evaluation  plan  is 
developed . 

The  report  concludes  that:  (1)  the  projects  results  will  not 
provide  statistically  valid  estimates;  (2)  States  are  not 
consistently  following  project  guidelines,  which  may  bias  the 
projects'  results;  (3)  reliability  of  data  from  the  continuing 
disability  review  (CDR)  phase  of  the  projects  is  questionable, 
since  the  States  have  had  little  experience  with  the  new  pro¬ 
cess;  and  (4)  evaluating  claimant  satisfaction  will  be  difficult 
and  costly,  and  has  not  been  adequately  addressed  in  the 
contractor's  survey  plan. 

Aboiit  the  first  conclusion,  given  the  variation  among  States,  no 
simple  random  sample  of  five  States  could  adequately  represent 
the  Nation.  We  have  attempted  to  choose  States  which  represent 
the  range  of  characteristics  of  all  States  in  the  Nation.  The 
accuracy  of  nationwide  projections  based  on  a  sample  of  five 
States  will  depend  on  how  varied  the  results  are  among  the 
project  States. 

Concerning  the  consistency  of  States  in  implementing  the 
projects,  we  believe  all  of  the  participating  States  are 
following  the  basic  project  guidelines  for  processing  demon¬ 
stration  cases.  These  projects  are  demonstrations  of  how  the 
f ai-e-tr - f ace  interview  process  would  work  under  the  existing 
Federa 1 -State  arrangement,  in  which  the  States  have  a  certain 
amount  of  flexibility  to  administer  and  manage  the  State 
disability  determination  services.  Thus,  the  variables  that 
exist  reflect  the  fact  that  the  demonstration  projects  must 
operate  within  the  constraints  of  each  particular  State. 
Reasonable  efforts  have  been  made  to  use  examiners  with  similar 
skills  and  expertise  to  process  both  demonstration  and  control 
cases.  Our  data  collection  and  analysis  will  consider  all  of 
these  variables,  and  we  are  confident  that  sufficient  controls 
are  in  place  to  permit  an  adequate  assessment  of  the  project. 
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About  the  third  General  Accounting  Office  (GAO)  conclusion  listed 
above,  we  are  aware  of  the  "learning  curve"  associated  with  both 
the  demonstration  process  and  the  CDR  hearings  process.  In  fact, 
it  may  be  advantageous  that  the  hearings  process  is  not  well 
established  in  State  operations  since  both  processes  will  be 
going  through  a  development  stage  and  face  similar  learning 
curves.  Thus,  the  processes  may  be  more  comparable  than  would  be 
the  case  in  other  circumstances.  We  will  work  closely  with  OIG 
to  evaluate  the  extent  to  which  the  learning  curve  impacts  key 
outcome  measures  in  this  project.  The  process  evaluation  will 
provide  SSA  with  valuable  information  about  changes  in  the 
operations  of  these  processes  over  time. 

Concerning  measurement  of  client  satisfaction,  as  mentioned 
above,  we  are  now  uncertain  whether  such  an  evaluation  will  take 
place.  We  share  GAO's  concerns  that  evaluating  claimant  satis¬ 
faction  will  be  difficult  and  costly,  but  we  will  explore  ways  of 
determining  client  satisfaction  within  the  evolving  project 
evaluation  plan. 

GAO  Recommendat ion 

Require  that  all  States  implement  the  demonstration  projects 
consistently  or,  if  such  consistency  is  not  practical,  require 
that  the  evaluation  contractor  account  for  such  inconsistencies 
in  the  evaluation. 

SSA  Comment 

We  agree  that  accounting  for  any  inconsistencies  in 
implementation  of  the  projects  is  important.  The  projects  will 
be  evaluated  at  the  State  level,  taking  into  consideration  the 
particular  operating  procedures  and  methods  in  each  State.  As 
indicated  in  the  general  section  above,  we  will  consider  any 
operational  differences  in  the  evaluation. 

GAO  Recommendat ion 

Evaluate  the  need  for  additional  resources  for  States 
part  lcipat mq  in  the  demonstration  projects. 

SSA  Comment 

We  concur ,  and,  where  necessary,  we  have  allocated  additional 
funds  or  made  appropriate  workload  adjustments. 

GA'.  Recommendation 

Identify  and  consider  alternative  data  collection  approaches  for 
measuring  laimants’  satisfaction,  given  the  costs  for  in-person 
interviews  and  the  difficulty  in  separating  claimant  satisfaction 
with  the  process  from  satisfaction  with  the  decision. 
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SSA  Comment 

As  indicated  in  the  general  section  above,  we  are  now  considering 
whether  to  include  measurement  of  client  satisfaction  in  the 
overall  evaluation  of  the  projects. 

Other  Matters 


In  several  sections  of  the  report,  references  are  made  to  the 
examiner  making  the  final  decision  following  the  face-to-face 
interview,  with  the  implication  that  the  examiner  is  the  sole 
decisionmaker.  This  is  incorrect.  Following  the  interview,  the 
examiner /physician  team  makes  the  decision  on  demonstration 
cases . 

The  report  also  notes  (page  17)  that  SSA  project  staff  indicated 
that  additional  States  may  be  selected  to  participate  in  the 
project.  We  believe  the  States  that  are  now  participating  will 
provide  a  good  test  of  the  demonstration  procedures,  and  we  have 
no  plans  to  include  additional  States. 

SSA  issued  an  interim  report  to  Congress  (copy  attached)  on 
March  3,  1987,  which  explained  why  the  start  of  the  projects  had 
been  delayed  and  described  the  various  activities  undertaken  to 
date  in  preparation  for  full  implementation. 

Attachment : 

Interim  report 


(IMSMl 
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